Skip to content

Sense contention on macOS and Windows, not just on Linux - #12

Merged
MuteJester merged 1 commit into
mainfrom
parallelism/contention-on-mac-and-windows
Nov 15, 2026
Merged

MuteJester merged 1 commit into
mainfrom
parallelism/contention-on-mac-and-windows

Conversation

@MuteJester

Copy link
Copy Markdown
Owner

beam_available_cores() read /proc/loadavg inside #if defined(__linux__) and
nothing else. On macOS and Windows — two of the three platforms BEAM ships wheels
for — the runnable count was therefore always unknown, which means "assume no
contention", which means clause 3 planned eight threads whatever the machine was
doing. Phase 0's catastrophe, 301x slower than serial on a saturated node, was
defended against on one platform of three.

The obvious answer was already ruled out

Both platforms have getloadavg(). avail.h carries the measurement that
disqualifies it:

competitors loadavg[0] nr_running
0 0.71 3
8 1.99 11
12 3.70 16

half a second after twelve spinners exited it still read 9.22 … A per-call
decision built on the average would serialise every call for a minute after
BEAM's own previous call finished.

So these platforms need a different signal, not a portable spelling of the same
one.

What they do instead

competitors = ceil((system_busy_ns - own_ns) / wall_ns), clamped to allowed

macOS reads host_statistics(HOST_CPU_LOAD_INFO), getrusage(RUSAGE_SELF) and
clock_gettime(CLOCK_MONOTONIC). Windows reads GetSystemTimes,
GetProcessTimes and the performance counter. One syscall each, no sleep — the
cost budget allows nothing more, since this runs at every entry point with no
caching and Linux answers in microseconds.

own_ns is subtracted, and that is the whole design. Without it, a window
spanning BEAM's own eight-thread region reads as eight cores busy, so every call
would narrow the next because of the previous one — the same feedback loop the
load average was rejected for. Linux escapes it for free: nr_running is
instantaneous, so by the time it reads, BEAM's workers have exited. A sampled
estimate has to subtract its own usage to measure the same quantity.

Three state rules, kept testable

They live in beam_competitors_from_samples, not in platform code, so they can be
pinned on any machine:

  1. A thread's first reading is only a baseline.
  2. A window below 20 ms reports unknown and leaves the baseline to age.
    Replacing it per call is the bug that would make the whole mechanism useless —
    a tight loop of short calls would restart the window and never reach 20 ms, on
    exactly the workload that needs it.
  3. A reading that moves backwards resets rather than subtracting into an unsigned
    underflow.

Every failure path returns -1, which is what these platforms returned on every
call before. This cannot be worse than what it replaced; it can only start being
right sooner.

Honest about being second best

It measures a window rather than an instant, so it lags where nr_running does
not. The lag lands on the safe side — planning narrow costs about 2x, planning
wide cost up to 3814x — which is also why a fraction of a core rounds up. Two
things it cannot do, both documented: system CPU time covers every CPU while
allowed may be a subset (the runnable count makes the same approximation), and a
busy fraction saturates at 100% where a count does not, so it cannot tell 48 busy
cores from 48 cores with 200 runnable threads. Both land on available = 1, which
is the answer that matters.

Two defects CI would not have caught

Found by re-reading the code, not by a test — neither line is reachable from Linux:

  • Mach port leak. mach_host_self() hands out a send right on every call, and
    this runs at every entry point: one leaked port reference per BEAM call.
    Invisible to any test, fatal to a long-lived process. Now paired with
    mach_port_deallocate.
  • Performance-counter overflow. counter * 1000000000 overflows int64 after
    roughly fifteen minutes of uptime at the usual 10 MHz. A CI runner lives
    about that long, so the obvious one-liner passes green here and returns nonsense
    on a developer's machine. Now split into seconds plus remainder.

Two stale claims corrected

avail.h credited "the policy's own halving" with supplying the conservative
fallback. plan.c stopped halving when the ceiling form replaced it; the ceiling
does that work now. And test_avail.c said "no /proc, as on Windows" while
describing the unknown path as "not a special case anywhere" — that was the gap,
sitting in a comment.

Verified

  • C suite: 450 passed, 0 failed (439 + 11 new)
  • Python suite: 1241 passed, 2 skipped
  • Gate: exit 0, no FAIL and no note lines; every recorded number
    byte-identical — 78 numeric keys, 58 bit-identical math values, the saved-model
    sha256, thread independence, 416 exact quantities
  • Extension rebuilt and the hash moved: a2cc902b… → 334d37e9…
  • Four rules mutation-tested: advancing the baseline on a short window, dropping
    the backwards guard, dropping the own-time subtraction, and rounding down each
    fail exactly the test that names the property
  • Linux behaviour unchanged: available still tracks instantaneously (15 then 14
    of 16 cores between two reads)

Baselines moved by inventory only — c_cases.txt 439 → 450 and c_symbols.txt
261 → 263 — with a RERECORDED.md section naming all thirteen lines.

The macOS and Windows readers do not compile on the machine that wrote them.
CI is their first build and their first run, which is the point of having three
platforms in it.

beam_available_cores() read /proc/loadavg inside #if defined(__linux__) and
nothing else, so on macOS and Windows -- two of the three platforms BEAM
ships wheels for -- the runnable count was always unknown. Unknown means
"assume no contention", so clause 3 planned eight threads whatever the
machine was doing. Phase 0's catastrophe, 301x slower than serial on a
saturated node, was defended against on one platform of three.

The obvious portable answer is already ruled out by the measurements in
avail.h: the one-minute load average is wrong by -6 at eight competitors,
and half a second after twelve spinners exited it still read 9.22 while the
runnable count had returned to 2. A plan built on it "would serialise every
call for a minute after BEAM's own previous call finished". Both platforms
have getloadavg(). It is not the answer.

So they difference two cumulative CPU-time readings instead:

    competitors = ceil((system_busy_ns - own_ns) / wall_ns), clamped

macOS reads host_statistics(HOST_CPU_LOAD_INFO), getrusage(RUSAGE_SELF) and
clock_gettime(CLOCK_MONOTONIC); Windows reads GetSystemTimes,
GetProcessTimes and the performance counter. One syscall each and no sleep,
which the cost budget demands: this runs at every entry point with no
caching, and Linux answers in microseconds.

own_ns is subtracted, and that is the whole design. Without it a window
spanning BEAM's own eight-thread region reads as eight cores busy, so every
call would narrow the next BECAUSE OF THE PREVIOUS ONE -- the same feedback
loop the load average was rejected for. Linux escapes it for free, the
runnable count being instantaneous: by the time it reads, the workers have
exited. A sampled estimate has to subtract its own usage to measure the same
quantity.

The three state rules live in beam_competitors_from_samples rather than in
platform code, so they are testable on any machine. A thread's first reading
is only a baseline. A window below twenty milliseconds reports unknown and
leaves the baseline to age -- replacing it per call is the bug that would
make the mechanism useless, since a tight loop of short calls would restart
the window and never reach twenty milliseconds, on exactly the workload that
needs it. A reading that moves backwards resets rather than subtracting into
an unsigned underflow.

Every failure path returns -1, which is what these platforms returned on
every call before, so this cannot be worse than what it replaced; it can
only start being right sooner. It is second best and says so: a window
rather than an instant, so it lags, and the lag lands on the safe side.
Planning narrow costs about 2x and planning wide cost up to 3814x, which is
also why a fraction of a core rounds up.

Eleven tests, four of them mutated against the real suite: advancing the
baseline on a short window, dropping the backwards guard, dropping the
subtraction of our own CPU time, and rounding down instead of up each fail
exactly the test that names the property.

Two defects in the first draft, found by re-reading rather than by a test,
and neither reachable from Linux. mach_host_self() hands out a send right
per call and was leaking one Mach port reference per entry point. And
counter * 1000000000 on the performance counter overflows int64 after about
fifteen minutes of uptime at 10 MHz -- longer than a CI runner lives, so it
would have passed green here and returned nonsense on a machine up since
Tuesday.

Two stale claims corrected on the way. avail.h credited "the policy's own
halving" with the conservative fallback; plan.c stopped halving when the
ceiling form replaced it, and the ceiling does that work now. And
test_avail.c said "no /proc, as on Windows" while describing the unknown
path as not a special case anywhere -- it was the gap, in a comment.

Verified: C 450 passed, Python 1241 passed and 2 skipped, gate exit 0 with
no FAIL and no note lines, every recorded number byte-identical, extension
rebuilt from a2cc902b to 334d37e9. The macOS and Windows readers do not
compile on this machine; CI is their first build and their first run.
@MuteJester
MuteJester merged commit 41e9dbb into main Nov 15, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant